1. Technical Field of the Invention
The present invention relates generally to the image processing field; and, more particularly, to a method and apparatus for removing image artifacts from images that are produced by a periodically varying light source.
2. Description of Related Art
In a CMOS (Complementary Metal Oxide Semiconductor) active-pixel image sensor, a rolling shutter is commonly used to provide exposure control. A rolling shutter may be implemented by sequentially resetting each row of sensors in an image sensor array and sequentially reading the values of each row at a later time. The exposure period is equal to the interval between reset and read out, and the frame period is the time required to read the values of every row in the array.
FIG. 1 is a diagram that schematically illustrates the operation of a rolling shutter to assist in providing a clear understanding of the present invention. Consider an image data array having 100 rows (for clarity, only 9 rows are illustrated in FIG. 1). Assume that the time required to read out a single row, i.e., the row period, is T; and that the exposure period, illustrated by arrow 10 in FIG. 1, is set to n*T. The exposure of row 1 will begin at time t=0, and the row will be read out at time t=n*T. The exposure of row 2 will begin at time t=T, and row 2 will be read out at time t=(n+1)*T. Row 100 of the array will begin exposure at time t=99*T, and that row will be read out at time t=(n+99)*T.
One characteristic of a rolling shutter is that the exposure for each row occurs over a different time interval as is shown in FIG. 1. Accordingly, if the illumination of a scene varies with time, the brightness of the captured image will vary from row to row. If the period of the variation in the illumination level of the scene is longer than the integration period of the rolling shutter and shorter than the frame period, the variation will result in the appearance of image artifacts in the form of horizontal bands in the image. The number of bands in the image will be equal to a flicker period divided by the frame period.
These variations in the illumination level of a light source over time are known as “flicker”, and the horizontal bands in the image are referred to as image artifacts or as a “flicker pattern”. Fluorescent lamps, for example, have very strong flicker. The phosphors in the lamp are excited at each peak of the power line frequency and decay between peaks. The light pulses that are produced have a flicker frequency that is twice that of the AC power line. Incandescent bulbs do not exhibit flicker because the thermal masses of their filaments are sufficiently large to prevent the filaments from cooling significantly between AC peaks. The effect of flicker on the output of a CMOS image sensor is illustrated in the schematic image 14 of FIG. 2 in which the periodic horizontal bars 16 are quite apparent and significantly detract from the overall quality of the image.
The number of horizontal bars in an image such as shown in FIG. 2 is equal to the ratio of the frame period to the flicker period. The amplitude of the horizontal bars depends on characteristics of the particular light source and on the integration period of the rolling shutter of the CMOS image sensor.
Several methods have been devised to remove frame-to-frame flicker. For example, U.S. Pat. No. 5,793,886 discloses a method in which the histograms of different image frames are compared so that the variation in the mean illumination level as a function of time can be removed. The method described, however, does not apply to a situation wherein only a single image frame is available and wherein the flicker is manifested as a series of horizontal bands in the resulting image.
In other known methods to remove frame-to-frame flicker, the light level of the illuminant is monitored during the frame exposure. For example, in the method described in U.S. Pat. No. 6,208,433; a separate sensor is used to monitor the fluctuations of the illuminant; and the output of this sensor is used to correct the image. In U.S. Pat. No. 5,960,153, a linear image sensor is used to both capture the image and to measure the illuminant variation. The illuminant variation is used to control the exposure timing so that flicker does not appear in the final image.
In yet another known method, flicker is avoided by restricting the exposure period to integer multiples of the flicker period. Under such circumstances, each row integrates light over an integer number of complete cycles of the illuminant's fluctuation. As a result, the integrated light level is caused to be the same for each row, and bands are prevented from appearing in the image. A disadvantage of this method is that the exposure period cannot be reduced below the flicker period. If the illuminant is very bright, and the exposure period cannot be reduced; the resulting image will be overexposed. Such a problem may occur when a scene is illuminated with bright fluorescent lights or when the camera is pointed directly at a fluorescent light.
In general, known methods for removing frame-to-frame flicker from an image require several successive image frames for processing, require that the flicker be independently monitored, or require additional inputs. Accordingly, known methods for removing flicker are not fully satisfactory. What is needed is a method and apparatus for removing flicker produced by a periodically varying light source that operates on a single image, that does not require any additional input, other than values of the flicker and frame periods, and that does not require that the flicker be independently monitored. What is also needed is a method and apparatus that can be effectively used to capture images with greatly reduced flicker even under very bright fluorescent illumination.